In the latter half of the twentieth century, there began a phenomenon known as the information revolution. While the information revolution is a historical development broader in scope than any one event or machine, no single device has come to represent the information revolution more than the digital electronic computer. The development of computer systems has surely been a revolution. Each year, computer systems grow faster, store more data, and provide more applications to their users.
A modern computer system typically comprises one or more central processing units (CPU) and supporting hardware necessary to store, retrieve and transfer information, such as communication buses and memory. It also includes hardware necessary to communicate with the outside world, such as input/output controllers or storage controllers, and devices attached thereto such as keyboards, monitors, tape drives, disk drives, communication links coupled to a network, etc. CPU's (also called processors) are capable of performing a limited set of very simple operations, but each operation is performed very quickly. Data is moved between processors and memory, and between input/output devices and processors or memory. Sophisticated software at multiple levels directs a computer to perform massive numbers of these simple operations, enabling the computer to perform complex tasks, and providing the illusion at a higher level that the computer is doing something sophisticated.
Continuing improvements to computer systems can take many forms, but the essential ingredient of progress in the data processing arts is increased throughput, i.e., performing more of these simple operations per unit of time.
The computer is a sequential state machine in which signals propagate through state storing elements synchronized with one or more clocks. Conceptually, the simplest possible throughput improvement is to increase the speeds at which these clocks operate, causing all actions to be performed correspondingly faster.
Data must often be communicated across boundaries between different system components. For example, data may need to be communicated from one integrated circuit chip to another. In countless instances, an operation to be performed by a component can not be completed until data is received from some other component. The capacity to transfer data can therefore be a significant limitation on the overall throughput of the computer system. As the various components of a computer system have become faster and handle larger volumes of data, it has become necessary to correspondingly increase the data transferring capability (“bandwidth”) of the various communications paths.
Typically, a communications medium or “bus” for transferring data from one integrated circuit chip to another includes multiple parallel lines which carry data at a frequency corresponding to a bus clock signal, which may be generated by the transmitting chip, the receiving chip, or some third component. The multiple lines in parallel each carry a respective part of a logical data unit. For example, if eight lines carry data in parallel, a first line may carry a first bit of each successive 8-bit byte of data, a second line carry a second bit, and so forth. Thus, the signals from a single line in isolation are meaningless, and must somehow be combined with those of other lines to produce coherent data.
The increased clock frequencies of processors and other digital data components have induced designers to increase the speeds of bus clocks in order to prevent transmission buses from becoming a bottleneck to performance. This has caused various design changes to the buses themselves. For example, a high-speed bus is typically implemented as a point-to-point link containing multiple lines in parallel, each carrying data from a single transmitting chip to a single receiving chip, in order to support operation at higher bus clock speeds.
The geometry, design constraints, and manufacturing tolerances of integrated circuit chips and the circuit cards or other platforms on which they are mounted makes it impossible to guarantee that all lines of single link are identical. For example, it is sometimes necessary for a link to turn a corner, meaning that the lines on the outside edge of the corner will be physically longer than those on the inside edge. Circuitry on a circuit card is often arranged in layers; some lines may lie adjacent to different circuit structures in neighboring layers, which can affect stray capacitance in the lines. Any of numerous variations during manufacture may cause some lines to be narrower than others, closer to adjacent circuit layers, etc. These and other variations affect the time it takes a signal to propagate from the transmitting chip to the receiving chip, so that some data signals carried on some lines will arrive in the receiving chip before others (a phenomenon referred to as data skew). Furthermore, manufacturing variations in the transmitter driving circuitry in the transmitting chip or receiving circuitry in the receiving chip can affect the quality of the data signal.
Where bus clock speeds are relatively slow, data skew is not a significant concern. But as clock speeds increase, skew becomes relatively more significant. Eventually, the clock speeds become so fast that a first bit of a sequence transmitted on one line arrives at the same time as a succeeding bit of the same sequence transmitted on another line of the same link. In other words, the difference in transmission time is enough to equal the time between successive bits. Modern bus clocks can be expected to reach the point where skew can equal the time to transmit 10 or 20 successive bits. Moreover, skew is not constant. Skew and other variations in received signals can depend on operating temperature, supply voltages, and other dynamic factors.
Ideally, communications circuitry is tolerant of all these static and dynamic variations. With all these factors affecting the data signals transmitted on a transmission link, it is desirable to calibrate individual line circuitry to compensate for variations, and in particular, since critical parameters change over time, it is desirable to dynamically calibrate individual line circuitry while the digital data system is operating, i.e., while the link is available to transmit functional data.
One known technique for dynamic calibration involves the use of duplicate sets of certain receiver circuitry for each line of multiple parallel lines. In particular, adjustable analog circuits such as variable gain amplifiers, offset adders, and comparators may be duplicated for each line. The input analog signal is provided to both sets of receiver circuitry, allowing one set to be used for processing an incoming functional data signal and passing data through to registers or buffers which record the data, while the other set is being calibrated. While this approach enables dynamic calibration, it requires full duplication of considerable analog circuitry, significantly increasing the power consumption and the complexity of the device.
An alternative technique, disclosed in U.S. Pat. No. 6,606,576 to Sessions and in U.S. Pat. No. 7,072,355 to Kizer, is the use of a single additional redundant parallel line and associated receiver circuitry. A set of switches selects one line at a time for calibration, while the other lines are used to transmit functional data. This technique has certain limitations, not necessarily recognized, in its ability to handle large data skew, and specifically skew beyond a single bit time, the time required to transmit a single bit over a line. When a switch is made to enable one line for transmitting functional data and disable another line so it can be calibrated, it is desirable to do so without halting the link. However, if substantial data skew exists between the two lines being switched, there could be discontinuity at the output of the switch.
Conventional parallel interface designs either fail to adequately account for large data skew or other variations, or do so at the expense of substantial circuit complexity which can increase cost and/or power consumption at the interface, or require some interruption in data flow across the link. In order to support continuing increases in communications bus speeds, a need exists for improved communications circuitry which is tolerant of larger data skew without undue circuit complexity or interruption of function.